Building Jester News web app

The best way to learn something new in my field of web dev is to apply whatever you learn directly as soon as possible, and ask questions as early as possible. Mistakes are themselves learning points, bottlenecks in approaches or technologies chosen are a view into real-world cases you have to consider when proposing solutions.

Anyhow, I know I need to build something and carry myself with all the advantages gained being exposed to .NET apps from the lens of front end development and as a past student.

I intend to record in this post, as close as possible, the following:

Important decision points and reasons for any choice made
Errors encountered that are significant or present a new reality
Features added, and new things learnt
Milestones

Decisions made

Choosing a BE pattern

I’ve chosen MVC, it is simple and commonplace.

Update: It looks like I am approaching a MVVM style. However, this doesn’t change anything fundamentally yet as it is just a way to organise files according to their use cases: for pages, for partials, for blocks, for apis etc.
Choosing a ORM

I don’t have time to curate SQL queries for everything (or anything for that matter). I have chosen Entity Framework Core to complement the rest of the project.
Choosing a DB

I wanted to choose a DB that fits the following:
- Lightweight
- Free
- Preferably open source
- Availability through any host especially platforms like Vercel
- Very easy to integrate into ASP.NET Core and EF Core
The one that fit the bill was PostgreSQL. I shall be hosting it via Docker to keep things simple and easy to manage.
Choosing or configuring routes

Decision to be made as I go along. None decided beyond the boilerplate yet. Admittedly, the ease of scaffolding has me wondering if I have been doing things wrong all this time.
Choosing the product

As this is a BE-focused project, I needed something that can lead to creating several pages easily while attracting interactions. I had the idea to twist reality (or am I?), so I asked Bing Copilot to suggest a feed of parody news. Then I asked it to suggest a name for this app. It gave “NewsJester” as a suggestion but I am tweaking the order of the words. It is not too bad a name.

I will be asking copilot for all the sample data, and even testing its limits in being a co-contributor to the project - let’s see how that goes.
Choosing UI library / framework

This is easy. I shall remove the default bootstrap, jquery etc. They don’t serve any purpose for me. I will be replacing with TailwindCSS, and plain JS modules that I will bundle in.
Choosing to create a service class

My intention is to keep things minimal and simple, not make it clean from the get-go. I have suffered that burn of over-optimising too early before in another side project. Anyway, I have been mulling over the idea to extract the db context I have been using for getting news data in controllers to a data service.

I have seen services implemented and they looked a lot like repositories been injected as a dependency. I have been cautious so as to understand what makes one create services as opposed to just a larger data context class. I wanted to stay away from repositories as well because my previously-bamboozled self thinks it is a sign of a new pattern. Trying to stay clear of mixing things together.

You can tell I have decided to add data services, what are the reasons? Main reason is from a test perspective. I can not test things at the moment. Abstracting things will allow unit tests to be added and kept closer to implementation.

Let’s see what the difference between a service and a repository are, and why this isn’t heading in the wrong direction. I have written a TIL on it (think #15) but the key point was

A repository handles the data access and the service calls into it after performing any business logic needed.

In other words, a repository is where you set up direct CRUD communication with the database while the service uses and combines these CRUD methods exposed by the repository to run business-stipulated logic to fulfil requirements e.g. to transform data to a particular form, to validate against specific rules etc
Reason(s) for creating a separate service class

I have been enjoying the setup so far using 1 news service, 1 repository talking to the db, and 1 controller with many actions for news.

This meant it was easy to keep adding controller actions for endpoints etc. However, as soon as you move away from 1 root endpoint (aka controller), you will have to make choices again. Are you going to use the same service? The answer could be yes, and reasonably so. Sometimes, the answer will also be no because your new controller maybe controlling things that have nothing to do with your current service.

Now, what is the broad border between services? This is where I would think “service needs to stay on topic”, if it is heading to a different topic, then a new service should be created for that topic and all things that fall under it (even if there’s overlap).

This of course reminds me the principle of single responsibility. You aim for everything to be focused on just one topic, one task etc.

Now I have two services: news, and authors - I might have one for tags / categories depending on their use cases. The news service handles all things articles: list, search, create, update, delete, filter etc. The authors service will handle everything about authors such as their profile, list of all authors, topics they contributed to and possibly readership stats on their articles etc.

You see the distinction between news and authors, though they are interlinked. What they share in essence is data, but nothing else. The 1 repository is still in tact. When might we create another repository? possibly when we create another table of a different topic or connect to a different database.

First unpleasant feeling of implementing repositories and services

It is important to note that my driving force behind making decisions is there being a need for something as oppose to something being useful. In adding services, I found it is need for testing sufficiently. You can argue I don’t need to test yet which you’ll be correct on I am also aiming for something close to an enterprise app without being a perfectionist. Something good enough that just touches on important aspects like a newborn enterprise app would be. It has the right base, not the perfect finish.

That said, I felt uneasy, to say the least, trying to move search and pagination from controller to the service. Let’s take a look at what I had in the first instance

   // GET: NewsItems
   public async Task<IActionResult> Index(int page = 1, string? query = null)
   {
      const int pageSize = 10;
      List<NewsItem> newsItems = await _context.NewsItems.Where(a => a.Title.Contains(query ?? "")).ToListAsync();

      var totalItems = newsItems.Count;
      var totalPages = (int)Math.Ceiling((double)totalItems / pageSize);

      var model = new NewsListViewModel
      {
            NewsItems = newsItems.Skip((page - 1) * pageSize).Take(pageSize).ToList(),
            CurrentPage = page,
            TotalPages = totalPages,
            PageSize = pageSize,
            SearchTerm = query
      };

      return View(model);
   }

Now my idea was to move as much of whatever is perceived to be logic for pagination and search into the service, and keep the view model stuff in the control. That was my idea of separation. Imagine my shock when the view model started leaking in to the service. It couldn’t be as simple as just returning the filtered/searched and paginated result, we also needed to return total number of records the database holds for pagination UI to be accurate.

I knew now I needed to either invent a new model or make the service return a view model (NewsListViewModel) such that all the controller has to do is pass it to the view. This felt icky, is it even legal for a service to return a view model? So I thought. It started putting me off as I felt like I dirtied my process and code (again!). Time to seek counsel / answers.

With a bit of research, I learnt from the internet that you can create a new model that is just for the service’s return (as they call it, PaginatedResult). This eased a bit of my anxiety and I drifted towards implementing it this way. Later, I consult an enterprise project and saw that they just let the service return the view model directly, no need to create a middleman model. And so I shall follow the example of the giants before me.

   //NewsService
   public async Task<NewsListViewModel> SearchAndPaginateAsync(string searchTerm, int page = 1, int pageSize = 10)
   {
      List<NewsItem> newsItems = await _newsRepository.GetAllAsync();
      var totalItems = newsItems.Count;

      var filteredNews = newsItems.Where(a => a.Title.Contains(searchTerm ?? ""));

      if (filteredNews.Any())
      {
            var pagedSearchResults = filteredNews.Skip((page - 1) * pageSize).Take(pageSize).ToList();
            var totalPages = (int)Math.Ceiling((double)totalItems/ pageSize);

            return new NewsListViewModel
            {
               NewsItems = pagedSearchResults,
               CurrentPage = page,
               TotalPages = totalPages,
               PageSize = pageSize,
               SearchTerm = searchTerm
            };
      }
      else
      {
            return null;
      }
   }

And now the controller looks like this

// GET: NewsItems
   public async Task<IActionResult> Index(int page = 1, string? query = null)
   {
      const int pageSize = 10;

      var model = await _newsService.SearchAndPaginateAsync(query ?? "", page, pageSize);

      return model != null ? View(model) : View(new NewsListViewModel());
   }

First feeling of success

I just reached a moment where I needed to create a block that can call an action to get data. Yes, I am talking about ViewComponents. Not long ago, I was building a fresh Umbraco website with a seasoned BE senior dev. One scene stuck with me during our interactions.
- A bit of confusion over the purpose of similarly named folders: Components, Views, Blocks, Partials etc.
Shipping to prod 🚀

I need to deploy this now to complete the incredible journey I have had breaking through old barriers, walking familiar lanes and rocking the pathway to success. How will I deploy?

My requirements are really simple: free hosting, accessible domain, and can last more that a few hours testing. All major cloud providers are disqualified as a result, simply because I can’t be bothered to signup with a card.

I have done a bit more research and found that railway.app or render.com can meet my needs. I will be picking Railway for running the app via docker, it is one of its features - automatic detection and deployment.

I will also use Supabase for hosting a free version of the postgreSQL I need. Can’t have both DB and App on the same host as the credit will run out fast!

What is missing?
- Prepare dockerfile
- Test locally
- Setup accounts on Railway and Supabase
- Initialise
- Deploy code to git
- Let Railway do its thing
- Test Supabase connection
- Generate Railway domain
- App is live 🚀
  - Fix issue with port (fixed by adding PORT var to railway)
  - Fix issue with connection string (fixed by correcting env var name on railway)
  - Fix issue with seeding file not being available on the prod workdir (See note at the bottom of this section)
It was a good run, there has been a few hiccups about port assignment by Railway, updating Env Variables on Railway helped. Now it should all work right? Nope. Look at my variables list

Environment variables in railway.app deployment center

You’ll find different variations of connection string variables. The app is indeed live, and alive shall we say, it is just pulling a long face because it can't find connectionString. I have tried 3 possibilities there, and it is still complaining.

I will have to sleep and find out tomorrow.

[A day later]

It was just a spelling mistake! It was meant to be ConnectionStrings__DefaultConnection (strings, not string).

Anyway, there is a second issue now where it isn’t able to find the seeding data. The file (sample-news.json) at the root of the project is not being copied to the /app folder that should serve all assets. Having to manually copy it in the dockerfile.

[Several days and hours later]

This has taken a whole lot of time to try myself with my best research skills. After significant amount of time chasing hints, testing locally and via docker, running into https issues. I decided I’ll stop giving this time, and move away to build more feature starting with Auth. The seeding file is not very critical in the sense that I can seed the db separately (locally in fact) and the prod version just needs to connect to a db with data.

[Editing days into the future]

I have had to detour and assume I don’t know anything about Docker to remove any bit of junk self-confidence leading me to the wrong alleys. I have fixed it and written the detour here

Errors

C’mon Postgres

Postgres complains about mismatch between its DateTime and C#‘s DateTime. This seems to have been a pain for decades.

How did I resolve it? I used DateTimeOffset as type, and remove the data annotation/validation from the property. [Spoiler alert: I didn’t fix it]
Why C#, why did you have to do this

I ask AI to autogenerate models for me to save time, and it suggested List<string> for tags. Lo and behold, EF Core complained it is not compatible with postgres. The fix? The suggested fix was to serialize to string and deserialize as you feel like it. It worked fine even when I typed in C# models to seed the database. It

At the end, it looked impossible or I was just incredibly stupid.

I came to the idea of just removing some bits slowly until I found the thing that totally breaks it. And so went the debugging. It became clear to me the deserializer could not deserialize to List. I asked one last time the internet, and got the suggestion to use string[]. And this was the absolute medicine for this AI-induced trauma.
Where’s this going?

I just noticed the DateTime wasn’t being deserialised correctly either. Looking at the model, I am reminded that I set DateTimeOffset. This seems to be incorrect and I fear I have to return to error #1.

Let’s go anyway, what’s one more error?

I tried improve techniques again, and again. The best result I got was to initialise a DateTime value and do transform to Utc Kind in EF Core after deserialisation. This was a cause for foregoing the ability to Seed DB with correct parsed dates. I am disappointed but this will be one to tackle later (or after I get a hold of a senior C# master).
You want to go your own way?

I have just been renaming an a controller from NewsItems to News along with Views and other parts of the app referencing it.

I had an endpoint /NewsItems/Details/{id} which was also a scaffolded route that I wanted to switch to /News/{id}, and what did I do?
```
   [HttpGet("/{id}")]
   public async Task<IActionResult> Details(int? id)
   {
      if (id == null)
      {
            return NotFound();
      }
      //... and so on
   }
```
I was just relying on my slightly blurry memories of the past that an attribute will do the job here.

You can guess what happened. It shutdown the /News endpoint.

I scratched my head for a bit and then realised it is Route not the Http attribute that I should use. Now that I had a great idea, guess what happened after it didn’t work straight away?

This…
```
[Route("/")] //This is on the news controller
public async Task<IActionResult> Index()
{
   //...stuff
}
```
And that just toppled the home page! It did solve the puzzle though, so I could just adjust it for each action
```
[Route("/News")] //This is on the news controller
public async Task<IActionResult> Index()
{
   //...stuff
}

[Route("/News/{id}")]
public async Task<IActionResult> Index(int? id)
{
   //...stuff
}
```
Strange feeling how I mixed them up but a good learning point nonetheless.
Templates can be used only with … ? A simple error triggered by modifying a
```
Html.DisplayFor(model => model.ToString().Split(","));
```
and that is how I learnt about Razor templates’ restriction on ‘complex’ expressions.

Features

After having set up everything (controllers, models, views, ef core, postgres) and got it to the point of showing all news articles, I asked AI for suggestions on how to improve and showcase more back end capabilities. Initially this was posed to DeepAI, and the answers were very impressive but I lost the result after I accidentally closed the (incognito) browser. I asked Copilot later, and here’s its response along with a new query of DeepAI as well. They look like they were trained on the same data.

I think it is a good time to end this page here as I focus on adding more enriching and challenging features. See you on the next page at Building Jester - Part 2

AI

These are the responses I received after querying “AI” agents Bing Copilot, and Deep AI for advise on what to build for this app. Some of their advice will be taken and others ignored. However, I will keep their responses here to demonstrate their capability with this regard, and how it accelerates prototyping for a less business-minded learner.

Copilot's response to backend features

Certainly! Building a parody news app sounds like a fun project. Here are some backend features you can consider implementing to showcase your development capabilities:

Authentication and Authorization:
- Implement user authentication using OAuth, JWT, or other authentication mechanisms.
- Define roles (e.g., admin, editor, reader) and restrict access to certain features based on user roles.
RESTful API Endpoints:
- Create well-designed API endpoints for CRUD (Create, Read, Update, Delete) operations on news articles.
- Include endpoints for searching, filtering, and sorting articles.
Database Design and Entity Framework Core:
- Optimize your database schema for storing news articles.
- Use Entity Framework Core to define models, relationships, and migrations.
- Consider using database indexes for performance.
Caching:
- Implement caching for frequently accessed data (e.g., article lists, popular articles).
- Use tools like Redis or in-memory caching to improve response times.
Background Jobs and Queues:
- Set up background jobs for tasks like sending email notifications or processing article updates.
- Use message queues (e.g., RabbitMQ, Azure Service Bus) for reliable communication between services.
Logging and Monitoring:
- Integrate logging (e.g., Serilog, NLog) to track application behavior and errors.
- Set up monitoring tools (e.g., Application Insights, Prometheus) to collect metrics and diagnose issues.
Error Handling and Validation:
- Implement robust error handling for API requests.
- Validate input data (e.g., request payloads) to prevent invalid data from reaching the database.
Testing:
- Write unit tests for your backend code using testing frameworks (e.g., xUnit, NUnit).
- Consider integration tests for API endpoints and database interactions.
Versioning:
- Design your API with versioning in mind (e.g., using URL versioning or headers).
- Handle backward compatibility gracefully when introducing new versions.
Security:
- Protect against common security vulnerabilities (e.g., SQL injection, cross-site scripting).
- Use HTTPS for secure communication.
- Consider implementing rate limiting to prevent abuse.

Remember that these features demonstrate your expertise in backend development. Choose the ones that align with your project goals and showcase your skills effectively. Happy coding! 🚀

Response from DeepAI

Sure, here are some features you could implement to showcase your backend development capabilities in your parody news app:

User authentication and authorization: Implement a user authentication system using JWT tokens or OAuth for users to login, register, and manage their profile. Ensure that only authorized users can access certain features or perform certain actions.
Search functionality: Implement a search feature that allows users to search for specific parody news articles based on keywords or categories. You could also include advanced search filters such as date range, author, or popularity.
Commenting and rating system: Allow users to comment on parody news articles and rate them based on their humor or accuracy. Implement a backend system to store and manage user comments and ratings.
Admin panel: Create an admin panel where designated administrators can manage parody news articles, users, comments, and ratings. Include features like article moderation, user banishment, and content management.
Notification system: Implement a notification system that alerts users of new parody news articles, comments, or ratings. Ensure that notifications are delivered in real-time and can be personalized based on user preferences.
Analytics dashboard: Provide administrators with an analytics dashboard that displays key metrics such as article views, user engagement, and popular topics. Use graphs and charts to visualize data and help administrators make informed decisions.
RESTful API: Develop a RESTful API that allows third-party developers to access and interact with your parody news app. Include endpoints for fetching articles, posting comments, and performing other CRUD operations.
Performance optimization: Optimize your backend code and database queries to improve the app’s performance and scalability. Implement caching mechanisms, query optimization techniques, and indexing strategies to reduce response times and enhance user experience.

By implementing these features, you can demonstrate your proficiency in backend development and impress users with a feature-rich parody news app. Good luck with your project!